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Patent Application Papers of: J.J. Garcia-Luna-Aceves and Chane Fullmer 

Attorney Docket No. NC30315 

TRANSMISSION-SCHEDULING COORDINATION AMONG COLLOCATED 

INTERNET RADIOS 

This application claims the benefit of U.S. Provisional Application No. 
60/230,395, filed September 6, 2000. 

FIELD OF THE INVENTION 

The present invention relates to the scheduling of transmissions 
without collisions in ad hoc networks with radio links in which routers can have 
both hosts and networks attached to them, and where some routers with 
wireless interfaces are collocated and receive conflicting transmission 
schedules from wireless neighbors. 

BACKGROUND 

Ad hoc networks (i.e., multi-hop packet radio networks) are computer 
networks in which routers are connected with wireless links. In ad hoc 
networks, nodes (stations or packet radios) can be mobile and communicate 
with one another either directly or through intermediate nodes, without relying 
on any pre-existing network infrastructure. 

Many medium-access control (MAC) protocols have been developed 
for wireless networks. These protocols can be classified as contention-based 
and schedule-based protocols. 

In a contention-based protocol, a node contends for access to the 
channel on a packet-by-packet basis. This is accomplished by either sending 
data packets to the channel or by means of a collision-avoidance handshake 
using small control packets. Examples of the former type of protocols are 
ALOHA, CSMA, BTMA, CSMA/CD. Examples of collision-avoidance 
protocols proposed to date include those disclosed in U.S. Pat. No. 5,319,64, 
U.S. Pat. No. 4,661,902, U.S. Pat. No.5,231,634, U.S. Pat. No. 5,502,724, 
and U.S. Pat. No. 5,721,725. Additional examples include, IEEE802.11, floor 
acquisition multiple access with non-persistent carrier sensing (FAMA-NCS), 
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receiver intitiated multiple access ( RIMA), and multiple access collision 
avoidance (MACA). 

Two key performance limitations of all contention-based protocols, 
including all collision-avoidance MAC protocols over single or multiple 
5 channels are that: (a) they cannot provide channel-access delay guarantees, 
and (b) they lack explicit support of collision-free multicasting or broadcasting. 

To provide delay guarantees and collision-free broadcasting and 
multicasting, network nodes can use a known transmission schedule or 
establish such a schedule dynamically to transmit data packets without 
10 collisions. Transmission schedules are established for time periods that are 

□ much longer than the duration of a single data packet or just a few data 
; j packets. In a transmission schedule, nodes are allowed to transmit at 

different times and on different data channels (e.g., frequencies, spreading 
codes, or their combination) in a way that no collisions occur. 

|5 The limitations of fixed transmission scheduling are the inability to 

* adapt to network changes and inefficient use of the channel if nodes are 
hj bursty sources of traffic. 

□ There are many approaches in the prior art based on dynamic 
transmission scheduling methods in which stations use ALOHA, slotted 

20 ALOHA or other contention-based protocols in an uplink to request time slots 
from a base station. An example of this approach is the system disclosed in 
U.S. Pat. 5,638,371. A number of protocols have been proposed in the recent 
past to provide dynamic time-slot allocation without requiring central base 
stations. These protocols can be classified as topology-independent and 

25 topology-dependent time scheduling protocols. 

Shepard, "A Channel Access Scheme for Large Dense Packet Radio 
Networks," SIGCOMM '96 Conference Proa, ACM 1996, "Scalable, Self- 
Configuring Packet Radio Network Having Decentralized Channel 
Management Providing Collision-Free Packet Transfer," US Patent 5,682,382, 
30 October 28, 1997; Chlamtac et al., Chlamtac, W. R. Franta, and K. D. Levin," 
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BRAM: The Broadcast Recognizing Access Method," IEEE Trans. Commun., 
vol. COM-27, pp. 1183-89, 1979, "Fair Algorithms for Maximal Link Activation 
in Multihop Radio Networks," IEEE Transactions on Communications, Vol. 
COM-35, no. 7, July, 1987, "Time-Spread Multiple-Access (TSMA) Protocols 

5 for Multihop Mobile Radio Networks," IEEE/ACM Transactions on Networking, 
Vol. 5, no. 6, December, 1997; and Ju and Li, Ji-Her Ju, Victor O.K. Li, "An 
Optimal Topology-Transparent Scheduling Method in Multihop Packet Radio 
Networks," IEEE/ACM Transactions on Networking, Vol. 6, no. 3, June 1998, 
have proposed topology-independent time-scheduling protocols. The 

10 protocols proposed by Ephremides and Truong, A. Ephremides, T. Truong, 
"Scheduling Broadcasts in Multihop Radio Networks," IEEE Transactions on 

■ ; 3 Communications, Vol. COM-38, No. 4, April, 1990; Corson, C. Zhu, M.S. 

4 Corson, "A Five-Phase Reservation Protocol (FPRP) for Mobile Ad Hoc 

hj Networks," Proc. IEEE INFOCOM '98; Young, C. D. Young, "USAP: A 

;rf Unifying Dynamic Distributed Multichannel TDMA Slot Assignment Protocol," 
[apple87] U.S. Patent 4,661,902, April 1987; and Tang and Garcia-Luna- 
Aceves, Z. Tang and J.J. Garcia-Luna-Aceves, "Hop-Reservation Multiple 
Access (HRMA) for Multichannel Packet Radio Networks", Proc. IEEE IC3N 
'98: Seventh International Conference on Computer Communications and 

5S Networks, Lafayette, Louisiana, October 12-15, 1998 are examples of 
topology-dependent scheduling protocols, such that a node acquires a 
transmission schedule that allows the node to transmit without interfering with 
nodes one and two hops away from itself, and such that channel reuse is 
increased as the number of neighbors per node decreases. Robust 

25 Environmentally Aware Link and MAC) (REALM) is another example of 

topology-dependent transmission scheduling; in this protocol, control packets 
used to exchange transmission schedules are transmitted. 

A common feature of all the schedule-based protocols in the prior art 
consists of assuming that each node has a single radio interface to the 
30 wireless network, or that the radios used by a single node (e.g., a base 
station) operate in orthogonal channels, such as a downlink and an uplink 
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channel in the case of a base station. In practice, however, multiple radio 
interfaces may be required at a single node to connect to the appropriate 
nodes in its neighborhood by means of multiple radio transceivers and 
directional antennas. In addition, two or more nodes with a single transceiver 
5 may be located near one another and be connected through a wired interface 
or a wired LAN. We refer to all these cases as collocated nodes. Collocated 
nodes present a new problem for the establishment and maintenance of 
transmission schedules dynamically, because the schedules that they receive 
from other nodes over wireless channels may be in conflict with one another, 
10 simply because different nodes may have radio connectivity with different 
collocated nodes. 

O SUMMARY OF THE INVENTION 

JTJ The present invention consists of a method for collocated nodes 

o communicating over a first interface to agree on a conflict-free transmission 
m schedule among themselves, which they can then use to collaborate with 

neighbors accessed through a second interface, for example through wireless 
□ links in order to obtain collision-free transfers of unicast, multicast and 

broadcast packets over wireless channels, and channel access delay 

guarantees. The present invention makes the collocated nodes behave as a 
20 single virtual node for the purpose of establishing a consistent transmission 

schedule throughout the nodes of a multihop wireless network. 

An embodiment of the present invention is implemented as the 
Collocated Neighborhood Established Transmission Scheduling protocol 
(CONETS) because it enables the nodes of an ad hoc network that are 

25 collocated and linked through wired media (a LAN or a wired link) to compute 
a single collision-free transmission schedule among themselves that they can 
then use to coordinate with other neighbors over wireless links to obtain 
collision-free transmission. In an embodiment of the invention, CONETS is 
used in combination with a dynamic transmission-scheduling protocol for 

30 multihop wireless networks, such as Neighboring Established Transmision 
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Scheduling (NETS), described in commonly assigned U.S. Patent Application 
No. 09/418,899, filed October 15, 1999, and incorporated by reference herein. 

CONETS assumes that collocated nodes must generate a common 
schedule to be used for a synchronous wireless network in which 
5 transmission times are organized into frames divided into slots. The amount 
of synchronization required in such a wireless network is the same type of 
synchronization required in any network operating with frequency hopping 
radios, such as those designed to operate in ISM bands and commercially 
available today. 

10 A deterministic scheduling algorithm in CONETS allows all the 

^ collocated nodes connected together through a LAN or wired link to agree on 

4 the same transmission schedule after the reliable exchange of schedule 

packets with one another. Each collocated neighbor of a node acknowledges 
^ the schedule packet sent by the node; this acknowledgment can be sent 
\M either as a separate packet or as part of a schedule packet. Collocated nodes 
!\ exchange CONETS schedule packets during a frame to derive a common 
;-=j schedule that takes effect for the transmission of data over wireless channels 
during the following frame. In addition to schedule packets, collocated nodes 
also transmit hello packets to inform their collocated neighbors of their 
20 existence, without having to send long schedule packets. The transmission of 
CONETS depends on the definition of the frames used in the radio channel(s) 
of the ad hoc network only in that such packets must be sent within the 
duration of a given frame and result in transmission schedules used by nodes 
to transmit over the wireless channel(s) of the ad hoc network that take effect 
25 starting with the following frame. 

A CONETS schedule packet provides a summary description of the 
two-hop neighborhood of a node in terms of: all the known nodes in the two- 
hop neighborhood of the transmitting node, the incoming and outgoing 
collision-free links of the node that have already been scheduled, the time 
30 slots and data channels where new links with the node can be reserved, and 
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the time slots and data channels where the node will be listening while not 
active in scheduled links. A CONETS acknowledgment packet is sent in 
response to a CONETS schedule packet. A CONETS hello packet simply 
states the identifier of the sending node and refers to the last schedule packet 
5 sent by the node. A CONETS hello-response packet is sent to correct the 
sequence number used by the sender of a CONETS hello packet to indicate 
what the latest schedule packet is. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 illustrates an ad hoc network according to an embodiment of the 
10 invention; 

Fig. 2 illustrates a frame utilized to transmit packets according to an 
embodiment of the invention; 

Figs. 3A and 3B are flowcharts illustrating process steps of scheduling 
transmission of packets according to an embodiment of the invention; 

15 Fig. 4 illustrates the format of a schedule packet according to an embodiment 
of the invention; 

Fig. 5 illustrates the format of a hello packet according to an embodiment of 
the invention; 

Fig. 6 illustrates the format of a hello-response packet according to an 
20 embodiment of the invention; and 

Fig. 7 illustrates scheduling information available at a node according to an 
embodiment of the invention. 

DETAILED DESCRIPTION 

A system for the establishment of a common schedule among 
25 collocated nodes for collision-free unicast, multicast, and broadcast 
transmissions in ad hoc networks according to an embodiment of the 
invention is disclosed herein. In the following description, numerous specific 
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details are set to provide a thorough understanding of the present invention. 
However, it will be evident to those skilled in the art that these specific details 
need not be used to practice the present invention and alternative 
embodiments are possible. In other cases, well-known structures and 
5 circuitry have not been shown in detail to avoid unnecessarily obscuring the 
present invention. 

The present scheduling protocol disclosed according to an embodiment 
will be referred to as the Neighborhood Established Transmission Scheduling 
(CONETS) protocol, because it enables all nodes that are members of an ad 

10 hoc network and collocated over a wired link or local area network (LAN) to 
obtain the exact same transmission schedule and interact with other nodes in 
the ad hoc network as if they were a single node for the purpose of building 
transmission schedules dynamically. The scheduling rules followed by 
collocated nodes are such that no collocated node is allowed to transmit while 

is one of its collocated neighbors is attempting to receive data. 

The novelty of the present invention results from combining a novel 
deterministic algorithm for selecting the node that should be allowed to 
transmit in a given time, together with a novel reliable exchange of schedule 
information among collocated nodes over the wire or LAN used to 
20 i n t e rco n n ect th em . 

I. Basic Service and Architecture 

Figure 1 illustrates aspects on an exemplary ad hoc network with 
collocated nodes according to an embodiment of the invention. The ad hoc 
network depicted in the figure consists of a number of subnetworks 20, 30, 40, 

25 50, which provide an extension of the Internet through a number of internet 
radios (IRs) 100, 110, 120, 130, 140, 150, 160, 170, 180. Each IR 100-180 is 
a wireless router with an IP address and a MAC address. The ad hoc network 
attaches to the Internet 900 via an access point, called "AirHead," which 
comprises IR 1 10 interconnected to an Internet router 200 through local area 

30 network 40. 
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IR 100, 140, and 180 are collocated through LAN 30, and the 
transmission schedule that IR 100 communicates to its non-collocated 
neighbors (IR 160 and IR 130) must be the same as the schedule that IR 140 
communicates to its non-collocated neighbor IR 1 10, and IR 180 
5 communicates to its non-collocated neighbor IR 170. IR 100, 140, and 180 
agree on the same transmission schedule using the CONETS protocol. The 
packets that pertain to the CONETS protocol are exchanged among IR 100, 
140 and 180 over LAN 30 only. The channel access protocol used to support 
CONETS depends only on the transmission media of LAN 30. 

10 CONETS packets are sent asynchronously over the link or LAN 

interconnecting collocated nodes. However, for the purpose of deriving a 
transmission schedule to be used over a set of wireless channels, CONETS 
assumes that time over the wireless channel(s) used is slotted and that slots 
are grouped into frames. The duration of a slot and a frame is configured in a 

15 node. We also assume that multiple orthogonal data channels are available; 
these channels can be implemented by means of multiple frequency bands, 
direct-sequence or frequency-hopped spreading codes, or combinations of 
waveforms that combine such techniques. 

To describe the operation of CONETS, the term active scheduled link 
20 (ASL) is used to denote a reserved sequence of contiguous time slots with a 
specific start slot and an associated data channel, where a data channel can 
be a spreading code, a frequency hop sequence, a frequency band, a data 
rate, and combinations of these and other transmission parameters. 

Slots are allocated to ASLs on multiples of link units, where a link unit 
25 is the minimum number of contiguous slots that a non-empty ASL can require. 
Hence, the slot range of an ASL is a multiple of link units. Furthermore, the 
start slot of an ASL must be a number that is a multiple of link units. This is 
done to avoid orphan slots that cannot be allocated to any ASL. 
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We note that an ASL specified for unicast transmissions has a 
transmitter and a receiver, and an ASL for multicast or broadcast 
transmissions has a transmitter and multiple receivers. 

Nodes can be identified by their MAC addresses or any type of 
5 identifier allowing a node to denote any of its collocated neighbors 
unambiguously. In the description of the embodiment of the invention 
presented herein, we simply use the term node identifier to denote the 
identifier used among collocated neighbors. A node can have any number of 
collocated neighbors over one or multiple wired transmission media. The 
10 node uses CONETS over each interface with collocated nodes to agree with 
all such neighbors on a common transmission schedule, which makes all 
collocated nodes act as a single node for the purpose of scheduling 
transmissions with other nodes over wireless links of a multihop wireless 
network. 

15 Collocated nodes exchange schedule packets, acknowledgments to 

such schedule packets, hello packets that refer to prior schedule packets, and 
hello-response packets designed to correct the information being used by a 
collocated neighbor. 

In one preferred embodiment of the present invention, CONETS is 
20 used in combination with Netwrok Established Transmission Scheduling 
(NETS) and Robust environmentally Aware Link and MAC (REALM), which 
are described in commonly assigned U.S. Patent Applications No. 
09/418,899, filed October 15, 1999 and No. 09/248,738 filed February 10, 
1999, assigned to the Assignee of the present invention and incorporated 
25 herein by reference. In this embodiment, REALM is used to determine when 
NETS schedule packets are sent periodically by each node, depending on its 
two-hop neighborhood. According to REALM, time is divided into frames of a 
known number of slots, and each frame is assigned a number that is known 
throughout the network. As illustrated in Figure 2, the first few slots of each 
30 frame 200 are dedicated to the transmission of NETS schedule packets, and 
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such slots are called control slots 202. The rest of the frame 200 is used for 
the transmission of data; the slots in the remaining of the frame 200 are called 
data slots 204. CONETS packets are exchanged over a wired link or a LAN 
by collocated nodes during the time of the frame 200 assigned for the 
transmission of data over the wireless channels. The transmission of 
CONETS schedule packets 206 is accomplished using a channel access 
protocol suitable for the transmission media used to interconnect the 
collocated nodes; for example, if the LAN interconnecting the collocated 
nodes is an Ethernet, carrier sense multiple access with collision detection 
(CSMA/CD) is used for the transmission of CONETS packets over it. Figure 2 
illustrates the case in which two of the collocated IRs in LAN 30 of Figure 1 
send schedule packets 206 and one of them sends a hello packet 208 during 
a given frame; the figure also illustrates the fact that CONETS packets are not 
transmitted synchronously with respect to the frame assumed for the 
transmission of packets over the wireless channel available. 

Establishing schedules in CONETS is based on the following 
principles: 

(a) Each node must advertise its transmission schedule to all its collocated 
neighbors, so that they can agree on the same version of the schedule, and 
each collocated node is capable of processing CONETS packets received 
over a LAN or wired link while it sends or receives data packets over one or 
more wireless links. 

(b) Data from a source must flow without interference from other sources 
over a reserved ASL, until conflicts due to mobility, errors due to the physical 
link, or the end of the flow are detected. Because of possible hidden 
terminals, the receiver(s) of an ASL must tell potential sources that the ASL is 
reserved. 

(c) Links must be established over multiple available data channels. 
Because of possible hidden terminals, both sender and receiver(s) of a link 
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must decide that the intended ASL does not interfere with other ongoing 
ASLs. 

(d) Each node receives an explicit acknowledgment to its transmitted 
schedule from each collocated neighbor in a LAN or wired link. 

(e) ASLs proposed or established by neighbors accessed through a 
wireless link have precedence over proposed ASLs from any of the collocated 
nodes to another node. 

(f) Collocated nodes can communicate with one another over a wired 
transmission medium, with no need for establishing an ASL over a wireless 
channel. 

CONETS provides two basic services: maintaining the set of collocated 
neighbors of a node over a given wired interface, and establishing a common 
transmission schedule among all those collocated nodes. Using CONETS, a 
set of collocated nodes appear to their neighbors as a single node with 
multiple node identifiers and capable of receiving or transmitting over multiple 
orthogonal channels concurrently, without being able to both transmit and 
receive at the same time over any channel. 

Figure 3 shows a flowchart outlining a preferred embodiment of 
CONETS in which REALM and NETS are used to schedule the transmission 
of packets into the radio channel(s) of an ad hoc network. An important 
assumption in this embodiment is that a node can receive the 
acknowledgments (ACKs) to any CONETS schedule packet from its 
collocated neighbors within the same frame. 

II. Information Exchanged and Maintained in CONETS 

Figure 4 presents the format of a canonical CONETS schedule packet 
416, which illustrates the combination of the various types of information used 
in the present invention. Fields in the canonical format are assumed to be 
required fields. The type of information conveyed in the schedule packets 
used in the embodiment of the present invention is the same as schedule 
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information conveyed in NETS schedule packets. A CONETS schedule 
packet specifies the transmission schedule and neighbor information known to 
the node and consists of: 

a) The sequence number 400 and age 402 of the schedule packet. 

5 b) The address of a collocated neighbor 404 that should send an ACK for 
the packet; a broadcast address (i.e., "all collocated neighbors") is used when 
new schedule information is reported. 

c) A list of one or more outgoing ASLs 406, which specify ASLs used for 
the node to transmit to its neighbors; each such ASL is specified by: 

310 The node identifier assigned to the neighbor in the ASL 

j The start slot of the ASL 

The slot range occupied by the ASL 

The data channel used for the ASL 

The frames to live for the ASL (FTL) 
J5 A bit indicating if the ASL is established (0) or requested (1) 

A schedule priority ticket whose value is picked by the node 

d) A list of one or more incoming ASLs 408, which specify ASLs used for 
neighbors to transmit to the node; each such ASL is specified by: 

The node identifier assigned to the node by its neighbor in the ASL 

20 The start slot of the ASL 

The slot range occupied by the ASL 

The data channel used for the ASL 

The frames to live for the ASL (FTL) 

A request bit indicating if the ASL is established (0) or requested (1) 



12 



Attorney Docket No. NC30315 



A schedule priority ticket whose value is picked by the neighbor 

e) A list of zero or more idle slot ranges (ISR) 41 0, with each ISR 
specified by: 

The start slot of the ISR 

The slot range of the ISR 

The data channel used for the ISR 

A bit indicating if the node is listening on the ISR 

A schedule priority ticket whose value is picked by the node 

f) A list of one or more active one-hop neighbors 412, with each entry in 
the list consisting of: 

The MAC address of a neighbor 

The XLID given by the node to the link with the neighbor 
The RLID given by the neighbor to the link with the node 

g) A list of zero or more MAC addresses of one- and two-hop neighbors 
414. 

For simplicity, it is assumed that ASLs and ISRs are specified in an 
order defined by their start slots, with the ASL and ISR with the smallest start 
slot number going first in the respective list. 

Slot ranges are specified in terms of link units. The schedule priority 
ticket is a random number used by nodes to determine which requested ASL 
(i.e., an ASL with request bit set to 1 ) asking for slots and a channel that 
overlap at least partially with other requested ASLs should win. The preferred 
embodiment of the present invention would follow the simple rule that the 
proposed ASL with the smallest ticket value is assigned the slots and channel 
requested. 
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Outgoing and incoming ASLs sent in a CONETS packet are either links 
agreed upon by all the nodes in a two-hop neighborhood, or they can be 
proposed links requested by nodes. ISRs have lower priority than established 
or requested ASLs and are used for nodes to execute quick transactions and 
to indicate to nodes receiving a CONETS schedule packet the slot ranges and 
associated channels that can be used to request ASLs with a given neighbor. 

In the preferred embodiment of the present invention, an 
acknowledgment (ACK) to a CONETS schedule packet specifies the node 
identifier of the sender of the CONETS packet, the node identifier of the 
receiver of the CONETS packet, the sequence number of the CONETS 
packet, and a flag indicating whether the node sending the ACK agrees or 
disagrees with the schedule proposed in the CONETS packet. In an 
alternative of the preferred embodiment of the present invention, ACKs to 
CONETS schedule packets are included as part of a CONETS packet in a list 
of ACKs. In this case, each item in such a list consists of the sequence 
number of a CONETS packet, the node identifier of the sender of the 
CONETS packet being acknowledged, and a flag indicating a positive or 
negative acknowledgment. 

Figure 5 shows the canonical format of a CONETS hello packet 506, 
which specifies the node identifier of the sender of the packet 500, the 
sequence number 502 and the age of the packet 504. The sequence number 
of a hello packet is the same sequence number of the last schedule packet 
used to report changes in the assumed schedule. The age field specifies a 
timeout during which the schedule information is valid. 

Figure 6 shows the canonical format of a CONETS hello-response 
packet 608, which specifies the node identifier of the sender of the packet 
600, the node identifier of the intended recipient of the packet 602, and the 
sequence number 604 and age 606 that the sending node has previously 
received from the intended receiver of the packet. 
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The information a node maintains about its two-hop neighborhood 
consists of: 

(a) The MAC addresses and associated node identifiers of all its one-hop 
neighbors, including its collocated neighbors. 

(b) The ASLs advertised by all its non-collocated one-hop neighbors. 

(c) The ISRs advertised by all its non-collocated one-hop neighbors. 

The aggregate of ASLs constitutes the working schedule of the two- 
hop neighborhood of a node, and the ISRs constitute the choices the node 
should first try to use to request new ASLs with neighbors. All collocated 
neighbors agree on the same schedule for themselves. 

A node maintains a hello entry for each collocated neighbor specifying 
the node identifier of the neighbor and a timer; the value of the timer is 
updated with the reception of a hello or schedule packet from the 
corresponding collocated neighbor. 

III. Maintaining Collocated Neighbors 

A node detects the presence of its collocated neighbors over a LAN or 
wired link by means of schedule packets and hello packets transmitted over 
the medium used to interconnect the collocated nodes. A node maintains a 
hello entry for each collocated neighbor; each entry specifies the node 
identifier of a neighbor and an age timer that determines the remaining time 
that the collocated neighbor and its schedule information can be assumed to 
be valid. The age of each hello entry is reduced each predefined unit of time, 
and a node determines that a collocated neighbor is not reachable through a 
wired link or LAN when the hello entry for the collocated neighbor reaches a 
zero age. A node deletes a collocated neighbor from its data structures when 
the information for the neighbor ages out locally, that is, the node decrements 
the age of a collocated neighbor down to zero. 
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In the preferred embodiment, the unit of time used for aging equals one 
frame or multiple frames; this choice renders small age fields and permits a 
node to decrement the age of a collocated neighbor only once a frame or 
once every number of frames. A node maintains a hello counter for itself and 
an age counter for each collocated neighbor. A node then resets its hello 
counter to its maximum value each time it sends a hello or CONETS schedule 
packet. A hello packet specifies the node identifier of the sending node, the 
sequence number of the last schedule packet sent to its collocated neighbors, 
and an age field specifying the period of time for which the schedule 
information referenced in the hello is valid. 

Referring now to Fig. 3a, after initializing, 300, and at the beginning of 
each frame 302, the node decrements its hello counter and the age counter of 
each of its collocated neighbors, 304. In the preferred embodiment of the 
present invention, the CONETS protocol is used together with NETS and 
REALM, and a collocated node can only transmit a schedule packet during 
the control portion of a frame to report the schedule it agreed upon with its 
collocated nodes during the data portion of the previous frame. As Figures 3A 
and 3B show, REALM is used in CONETS to determine in which control slot a 
node can transmit its schedule to its non-collocated neighbors over the 
wireless channel 306, a node then processes the frame to receive and store 
schedule packets from other nodes 308-310. Next the schedule computation 
procedures of NETS are used to process the schedule packets, 312. 

A node with collocated neighbors, therefore, receives the schedule 
packets during the control portion of a frame from neighbors with which it has 
radio connectivity, processes all such schedule packets according to Etiquette 
rules 1 to 10 using the same procedures disclosed for the NETS, and sends a 
CONETS schedule packet over the wired link or LAN to its collocated 
neighbors during the data portion of the frame. During the date portion of the 
frame, a node with collocated nodes may receive CONETS schedule packets, 
hello packets, acknowledgment packets, and hello-response packets. The 
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schedule of a node can be modified by CONETS schedule packets and loss 
of connectivity with a collocated node. 

As Figure 3A illustrates, after processing the schedule packets, 312, 
and determining that the schedule has changed, 314, a node transmits a 
CONETS schedule packet, 320, only if there is sufficient time left in the frame 
to receive all the acknowledgments from its collocated neighbors, 320. If the 
node transmits the CONET packet, the hello counter is reset, 322. If the node 
does not transmit a CONET package, the process begins again at the next 
frame start, 302. For simplicity, the event corresponding to the loss of a 
collocated neighbor is not shown in Figure 3. However, this event can be 
interpreted in the pseudocode presented in Figure 3 as the reception of a 
virtual schedule from the collocated node that becomes disconnected 
containing no ASLs and ISRs. The reception of the node's hello at a 
collocated neighbor restarts the timer used by the neighbor to delete schedule 
information reported by the node. Accordingly, a node that has no schedule 
modifications during a given frame, 314, sends a hello packet during the 
frame, 318, if it has not sent any hello or schedule packet for a period of time 
proportional to the maximum age allowed for a collocated node to remain 
valid according to its hello counter, 316. 

The sequence number used to determine the most up to date schedule 
received from a collocated neighbor may need to be recycled. Even if a large 
sequence number space is used (e.g., 32 bits can be used to denote a 
sequence number), a node may need to recycle the sequence number it uses 
for its transmission schedule after it reboots following a failure. The following 
steps are followed to ensure that the most recent transmission schedule from 
a node is accepted as such by all its collocated neighbors. 

To start a safe recycling of sequence numbers in the presence of 
failures, a node that reaches the maximum allowed sequence number sends 
a hello with the maximum sequence number and an age of 0, which instructs 
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its collocated neighbors to prepare to accept a smaller sequence number for 
the node. 

Referring now to Figure 3B, therein are process steps used at a node 
to process hellos or CONETS schedule packets received from collocated 
nodes at 318 and 322, respectively, of Fig. 3A. 

A node that receives a hello packet, 334, from a collocated neighbor, 
through 324 and 334, that is determined new and not known previously, 336, 
transmits a Hello packet, 338, to the collocated neighbor even if it has no 
schedule changes to report; the Hello packet functions as a schedule packet 
that specifies the node identifier of the new collocated neighbor as the node 
that must acknowledge the schedule packet, and the sequence number of this 
schedule packet is the same as the last sequence number acknowledged by 
other collocated neighbors. 

A node that is started or rebooted is initialized with an empty 
transmission schedule and therefore sends a hello packet with a zero 
sequence number and the maximum allowed age. 

As shown in Figure 3, when node Y receives 340 a hello from a 
collocated neighbor X in which the sequence number used by the sending 
node X is smaller than the sequence number stored at the receiving node Y 
for the sending node, 340, then node Y sends a hello-response packet to 
node X specifying the sequence number and age locally available at node Y 
for its collocated neighbor X, 350. In turn, when node X receives through 324, 
334, a hello-response packet addressed to it, 352, it increases its sequence 
number to equal the maximum of its current sequence number and one plus 
the sequence number received in the hello-response packet from node Y, 
354; after that, node X sends a hello with the resulting sequence number, 
356. If a hello-response is not received, 352, node X, and an ACK to CONET 
packet is received, 358, node X marks node Y as having sent the ACK. If an 
ACK is not received, 358, node X and if more data slots are available to 
receive appropriate CONETS packets, 362, node X determines whether map 
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packets are to be received. Taking these steps ensures that node X uses 
sequence numbers for its hello and schedule packets that all its collocated 
neighbors can assume to be the most recent from node X. 

As shown in Figure 3, if node Y receives, through 324, 334, 336, 340, 
342, a hello with a zero (0) age and the maximum sequence number allowed 
from a collocated neighbor X, 344, it then locally assigns a zero (0) sequence 
number and a maximum age to node X, 346; this allows node Y to maintain 
the last schedule reported by X while it transmits a subsequent hello or 
schedule packet reliably with a sequence number equal to 1 , otherwise it 
assigns a sequence number equal to the sequence number in the hello, 348. 

As shown in Figure 3, when node Y receives, through 324, a CONETS 
schedule packet from a collocated neighbor X in which the sequence number 
used by the sending node X is smaller than the sequence number stored at 
the receiving node Y for the sending node, 326, then node Y discards the 
packet, 332 and sends a hello-response packet to node X with the sequence 
number stored for X at node Y, 350. If node Y receives a CONETS schedule 
packet otherwise, it updates its own schedule based on CONETS schedule 
packet using NET procedures, 328. 

IV. Establishing Common Transmission Schedules 

For convenience, we refer to all the data that must be transmitted by a 
node to one or multiple neighbors over a given ASL as a flow. Data packets 
in the same flow, therefore, can be addressed to different network-level 
destinations sharing the same relay next node. The scheduling of such 
packets over an ASL is outside the scope of this invention. However, it 
should be apparent to those skilled in the art that the ASLs established in 
CONETS tend to be longer lasting than individual connections, therefore, 
because they service multiple connections. 

Because nodes in a two-hop neighborhood may have inconsistent 
scheduling information and send out their requests for ASLs concurrently, 
more than one requested ASL sent by nodes during the same frame may 
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specify conflicting slot ranges, data channels or intended receivers. The 
establishment of a common transmission schedule for all collocated nodes 
using CONETS is based on a simple distributed election method based on a 
novel etiquette of channel reuse. In one preferred embodiment of the present 

5 invention, a node listens for REALM control packets during the first few slots 
of a frame and uses the rest of the frame to exchange CONETS schedule 
packets with its collocated neighbors; the common schedule agree upon by 
the node and all its collocated neighbors takes effect at the beginning of the 
next frame. In another preferred embodiment of the present invention, a node 

10 exchanges CONETS schedule packets with its collocated neighbors based on 
schedule information received during the prior frame from non-collocated 
neighbors, and the common schedule agreed upon by the node and its 
collocated neighbors takes effect at the beginning of the next frame. 

CONETS bases the scheduling of transmissions on a set of etiquette 
15 rules, which are an extension of the rules defined for the NETS protocol. The 
modifications to the NETS etiquette allow two or more collocated nodes to 
transmit or receive concurrently over orthogonal channels, and prevent a 
collocated node from transmitting while any of its collocated neighbors is 
receiving. The CONETS etiquette consists are the following: 

20 Etiquette Rule 1 : The schedule assumed by all the collocated nodes in the 
same LAN or wired link takes effect in the frame following the frame during 
which CONETS schedule packets are exchanged. 

Etiquette Rule 2: An incoming or outgoing ASL already established has 
precedence over any requested ASL that conflicts with the established ASL. 

25 Etiquette Rule 3: For any requested ASLs that conflict with one another, the 
following precedence rules must be observed: 

3a) ASLs received from non-collocated neighbors have precedence over 
ASLs received from collocated neighbors and ASLs requested by the node 
itself. 
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3b) ASLs for broadcast transmissions have precedence over ASLs for 
multicast or unicast transmissions. 

3c) ASLs for multicast transmissions have precedence over ASLs for 
unicast transmissions. 

5 3d) Among the ASLs with the same precedence due to the type of 
transmission, the ASL with the smallest schedule priority ticket has 
precedence. 

Etiquette Rule 4: The following precedence rules must be observed for nodes 
and collocated nodes that work in half-duplex mode: 

10 4a) No ASL to or from a given node may overlap on any time slot with 
another ASL to or from the same node. 

4b) No ASL from any collocated node may overlap any ASL to any of its 
collocated neighbors on any time slot. 

4c) ASLs from collocated node peers using identical channels must not 
15 overlap on any time slot. 

Etiquette Rule 5: The slots of an ASL must be contiguous and the same data 
channel must be used for the entire ASL. 

Etiquette Rule 6: ASLs have precedence over ISRs, and between two ISRs 
that use the same channel and overlap in at least one time slot, the ISR with 
20 the smallest schedule priority ticket has precedence. 

Etiquette Rule 7: During any given time slot assigned for data transmission, a 
node that is not transmitting or receiving on an established ASL must be 
listening in one of its advertised ISRs. 

Etiquette Rule 8: When node i needs to establish a new ASL with a neighbor 
25 j, it must choose one of the advertised ISRs from j without creating any 
conflict with any other etiquette rules. 

Etiquette Rule 9: A node can transmit only over established ASLs. 
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Etiquette Rule 10: Transmissions over an advertised ISR must be done using 
a listen-before-talk etiquette over the channel specified for the ISR. 

Etiquette Rule 1 1 : A node can announce a new ASL to or from the node itself 
torn its non-collocated neighbors only after all its collocated neighbors have 
5 agreed to include the ASL in the common transmission schedule maintained 
by all collocated nodes. 

All the valid ASLs constitute the working schedule of the node. The 
working schedule of node i is denoted by WS_i. The valid ISRs constitute the 
feasible schedule of the node. The feasible schedule of node i is denoted by 

10 FSj. Node i updates WSJ and FS_i when it receives a CONETS schedule 
packet from a collocated neighbor, or a schedule packet from a non- 
collocated neighbor. Schedule packets and CONETS schedule packets are 
received in different portions of a frame. In one preferred embodiment, the 
first few slots of a frame are dedicated to the exchange of schedule packets 

15 among neighbors with radio connectivity with one another. The rest of the 
frame can then be used over a wired link or LAN by collocated nodes to 
exchange CONETS schedule packets. Figure 2 illustrates the structure of the 
frame in such an embodiment 

To update WS_i, node i applies etiquette rules 1 to 1 1 on each of the 
20 updated ASLs or new ASLs reported in a CONETS schedule packet to 

determine which are valid ASLs among all those reported by its neighbors and 
the ASLs originated by the node itself. Given Etiquette Rule 1 1 , a node 
requiring to establish a new ASL must first announce the new ASL to its 
collocated neighbors in a CONETS packet and receive the positive ACKs 
25 from all its collocated neighbors, before it can announce the proposed ASL to 
its non-collocated neighbors. 



Consider the wireless network shown in Figure 1 . For simplicity, the 
system is assumed to have three orthogonal channels and a frame is shown 
30 as consisting of one control slot and 10 data slots. Figure 7 shows the 
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scheduling information available at IR 180, is labeled A. As the figure shows, 
node A has an established ASL with node B in channel 1 lasting for slots 1 
and 2. By means of CONETS schedule packets exchanged with its 
collocated neighbors IR 100 and IR 140, which are labeled B and C, 
5 respectively, node A also knows that there is an established ASL from node B 
to IR 160, which is labeled E, on channel 2 during slots 1 through 3, an 
established ASL from node C to IR 130, labeled F, in channel 3 during slots 4 
to 8, and a proposed ASL from node D to IR 1 10, labeled G, over channel 2 
during slots 9 and 10. 

io For IR 180, 100, and 140 to send new transmission schedules to IRs to 

which they connect via radio links, they must exchange CONETS schedule 
packets among themselves to agree on the next schedule during the data 
slots of a frame, which they can then use in the control slot(s) of the following 
frame. 

15 Hence, in this example, IR 100 cannot propose an ASL to IR 160 

before its collocated neighbors IR 180 and IR 140 agree on it. Again, the end 
result of using CONETS among collocated neighbors is that all collocated 
neighbors present exactly the same transmission schedule during a given 
frame to their non-collocated neighbors. 

20 Although the invention has been described in the context of particular 

embodiments, it should be realized that a number of modifications to these 
teachings may occur to one skilled in the art. Thus while the invention has 
been particularly shown and described with respect to these particular 
embodiments thereof, it will be understood that changes in form and scope 

25 may be made therein without departing from the scope and spirit of the 
invention. 
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CLAIMS 

What is claimed is: 

1 1 . A method for scheduling transmissions in a network including a 

2 plurality of collocated nodes and a plurality of non-collocated nodes, wherein 

3 the plurality of collocated nodes communicate between one another over a 

4 first interface and the plurality of non-collocated nodes communicate with the 

5 plurality of collocated nodes over a second interface, said method comprising 

6 the steps of: 



7 exchanging scheduling information between the plurality of 

8 collocated nodes over the first interface, said scheduling 

9 information associated with transmissions between the plurality 
io of collocated nodes and each of the plurality of non-collocated 
n nodes on the second interface; and 

12 determining, based at least in part on said scheduling 

13 information, a schedule for the plurality of collocated nodes for 

14 transmissions between the plurality of collocated nodes and 

15 each of the plurality of non-collocated nodes on the second 

16 interface. 

l 2. The method of claim 1 , wherein said step of exchanging scheduling 



2 information comprises exchanging said scheduling information over said first 

3 interface during a time frame while the plurality of non-collocated nodes send 

4 data packets to, and receive data packets from, each of the plurality of 

5 collocated nodes over the second interface. 

1 3. The method of claim 2, wherein said time frame comprises a first 

2 time frame and said step of determining comprises determining a schedule for 

3 the plurality of collocated nodes for transmissions between the plurality of 

4 collocated nodes and each of the plurality of non-collocated nodes on the 

5 second interface, wherein said transmissions occur during a second time 

6 frame subsequent to said first time frame. 
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1 4. The method of claim 1 , wherein communications over the second 

2 interface are performed using a plurality of time frames, each comprising a 

3 control and data portion, wherein said scheduling information comprises first 

4 scheduling information, wherein said step of exchanging scheduling 

5 information comprising exchanging said first scheduling information over the 

6 first interface during said data portion of said time frame, and wherein the 

7 method further comprises the step of exchanging second control information 

8 between the plurality of non-collocated nodes and each of the plurality of 

9 collocated nodes over the second interface during said control portion of said 

10 time frame, said second scheduling information associated with transmissions 
n between the plurality of collocated and each of the plurality of non-collocated 
12 nodes. 

1 5. The method of claim 1 , wherein said step of exchanging scheduling 

2 information comprises sending a schedule packet from each of a first at least 

3 one collocated node to each of a second at least one collocated node of said 

4 plurality of collocated nodes, said schedule packet including an indication of 

5 all known nodes in the two-hop neighborhood of each of said first at least one 

6 node , incoming and outgoing collision-free links of each of said first at least 

7 one node that are already scheduled, time slots and data channels in which 

8 new links with each of said first at least one node can be reserved, and time 

9 slots and data channels on which each of said first at least one node will be 

10 listening while not active in scheduled links. 



1 6. The method of claim 1 , wherein said step of exchanging scheduling 

2 information comprises the steps of: 

3 sending a schedule packet from a first at least one collocated 

4 node of said plurality of collocated nodes to a second at least 

5 one collocated node of said plurality of collocated nodes over 

6 the first interface; and 

7 sending, in response to receiving said schedule packet, an 

8 acknowledgement packet from said second at least one 
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9 collocated node to said first at least one collocated node over 

10 the first interface. 

1 7. The method of claim 6, wherein said step of exchanging scheduling 

2 information further comprises the steps of: 

3 setting, in said second at least one collocated node of said 

4 plurality of collocated nodes, a sequence number of a last 

5 received schedule packet to a value of a sequence number of 

6 said schedule packet received from said first at least one 

7 collocated node of said plurality collocated nodes in said step of 

8 sending; 

9 sending a hello packet from said first at least one collocated 

10 node of said plurality of collocated nodes to said second at least 
n one collocated node of said plurality of collocated nodes, said 

12 hello packet identifying said first at least one collocated node 

13 and a sequence number of a last sent schedule packet from 
H said first at least one collocated node; 

15 determining if said sequence number of a last sent schedule 

16 packet indicates that said sequence number of a last sent 

17 schedule packet is less than said sequence number for a last 

18 received schedule packet; 

19 and, in response to a positive determination: 

20 transmitting a hello-response from said second at least one 

21 collocated node to said first at least one node of said plurality of 

22 collocated nodes, said hello-response including said sequence 

23 number for a last received schedule packet. 

l 8. The method of claim 7, wherein said hello packet comprises a first 



2 hello packet and said step of exchanging scheduling information further 

3 comprises the steps of: 
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4 receiving said hello-response at said first at least one collocated 

5 node of said plurality of collocated nodes; 

6 resetting said sequence number of a last sent schedule packet 

7 to the larger of said sequence number of a last sent schedule 

8 packet or 1 plus said sequence number of a last received 

9 schedule packet received in said hello-response; and 

10 sending a second hello packet from said first at least one 

n collocated node said second at least one collocated node of said 

12 plurality of collocated nodes, said hello packet identifying said 

13 first at least one collocated node and said sequence number of 

14 said last sent schedule packet as reset in said step of resetting. 

1 9. The method of claim 6, wherein said step of exchanging scheduling 

2 information further comprises the steps of: 

3 setting, in said second at least one collocated node of said 

4 plurality of collocated nodes, a sequence number of a last 

5 received schedule packet to said value a sequence number of 

6 said schedule packet received from said first at least one 

7 collocated node of said plurality of collocated nodes in said step 

8 of sending; 

9 sending a hello packet from said first at least one collocated 

10 node of said plurality of collocated nodes to a second at least 
n one collocated node of said plurality of collocated nodes, said 

12 hello packet identifying said first at least one collocated node 

13 and a sequence number of a last sent schedule packet from 

14 said first at least one collocated node; 

is determining if said sequence of a last send schedule packet 

16 indicates that said sequence number or a last sent schedule 

n packet is less than said sequence number for a last received 

18 schedule packet; 
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19 and, in response to a negative determination, setting said 

20 sequence number of a last received schedule packet to said 

21 sequence number of a last sent schedule packet. 

1 10. The method of claim 1 , wherein said first interface comprises a 

2 wired link and said second interface comprises a wireless link. 

1 11. The method of claim 1 , wherein said wireless link comprises a 

2 plurality of RF channels and the plurality of collocated nodes communicates 

3 with at least two of the plurality of non-collocated nodes simultaneously over 

4 orthogonal channels of said plurality of RF channels. 

1 12. A communications network, said network comprising: 

2 a plurality of non-collocated nodes, each of said plurality of non- 

3 collocated nodes capable of receiving and transmitting 

4 transmissions on a first interface; and 

5 a plurality of collocated noes, said plurality of collocated nodes 

6 each capable of communicating between one another over a 

7 second interface, each of said plurality of collocated nodes 

8 further capable of receiving and transmitting transmissions to 

9 and from said plurality of non-collocated nodes on said first 

10 interface, wherein said plurality of collocated nodes exchanges 

11 scheduling information with one another over said second 

12 interface, said scheduling information associated with 

13 transmissions between the plurality of collocated nodes and 

14 each of the plurality of non-collocated nodes on said first 

15 interface, and determines, based at least in part on said 

16 scheduling information, a schedule for said plurality of collocated 

17 nodes for transmission between said plurality of collocated 

18 nodes and each of said plurality of non-collocated nodes on said 

19 first interface. 



28 



Attorney Docket No. NC30315 



1 3. The network of claim 1 2, wherein said plurality of collocated nodes 
exchanges said scheduling information over said second interface during a 
time frame while said plurality of non-collocated nodes send data packets to, 
and receive data packets from, each of the plurality of collocated nodes over 
said first interface. 

14. The network of claim 1 3, wherein said time frame comprises a first 
time frame and wherein said schedule determined by said plurality of 
collocated nodes comprises a schedule for said plurality of collocated nodes 
for transmissions between said plurality of non-collocated nodes on said first 
interface, wherein said transmissions occur during a second time frame 
subsequent to said first time frame. 

1 5. The network of claim 12, wherein communications over said first 
interface are performed using a plurality of time frames, each comprising a 
control and data portion, wherein said scheduling information comprises first 
scheduling information, wherein said plurality of collocated nodes exchanges 
said first scheduling information over said second interface during said data 
portion of said time frame, and wherein said plurality of collocated nodes 
further exchanges second control information with each of said plurality of 
non-collocated nodes over said first interface during said control portion of 
said time frame, said second scheduling information associated with 
transmissions between said plurality of collocated nodes and each of said 
plurality of non-collocated nodes. 

1 6. The network of claim 1 2, wherein said plurality of collocated nodes 
exchanges scheduling information comprising a schedule packet, said 
schedule packet including an indication of all known nodes in the two-hop 
neighborhood of a sending collocated node, incoming and outgoing collision- 
free links of said sending collocated node that are already scheduled, time 
slots and data channels in which new links with said sending collocated node 
can be reserved, and time slots and data channels on which said collocated 
node will be listening while not active in scheduled links. 
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1 1 7. The network of claim 1 2, wherein said first interface comprises a 

2 wireless link and said second interface comprises a wired link. 

1 18. An apparatus in a communications network including a plurality of 

2 non-collocated routers capable of communicating over a first interface, said 

3 apparatus comprising: 

4 at least two collocated routers, said at least two collocated 

5 routers capable of communications between one another over a 

6 second interface, and said at least two collocated routers 

7 capable of communications with each of the plurality of non- 

8 collocated routers over the first interface, wherein said at least 
□ 9 two routers exchange scheduling information over said second 

io interface, said scheduling information associated with 

j In transmissions between said at least two collocated routers and 

1^3 12 the plurality of non-collocated routers on the first interface, and 

U113 wherein said at least two collocated routers determine, based at 

j\ 14 least in part on said scheduling information, a schedule for 

Qi5 transmission between said at least two collocated routers and 

il\6 each of the plurality of non-collocated routers on the first 

;::fi7 interface. 

1 19. The apparatus of claim 18, wherein the first interface comprises a 

2 wireless link and said second interface comprises a wired link. 

l 20. The apparatus of claim 19, wherein said at least two collocated 



2 routers exchange said scheduling information over said wired link 

3 substantially simultaneously with said at least two collocated routers 

4 exchanging data with the plurality of non-collocated nodes over said wireless 

5 link. 

1 21 . The apparatus of claim 18, wherein said at least two collocated 

2 routers and the plurality of routers and the plurality of non-collocated routers 

3 communicating over the first interface using a plurality of time frames, each of 

4 said plurality of time frames including a control portion and a data portion, and 
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5 wherein said at least two collocated routers exchange said scheduling 

6 information over said second interface substantially simultaneously with the 

7 data portion of a selected at least one frame of said plurality of frames 

8 transmit on the first interface. 

1 22. The apparatus of claim 21 , wherein said selected at least one frame 

2 comprises a first selected at least one frame and wherein said schedule for 

3 transmission between said at least two collocated routers and each of the 

4 plurality of non-collocated routers on the first interface comprises a schedule 

5 for transmission during a second selected at least one frame of said plurality 

6 of frames transmit subsequent to said first selected at least one frame. 
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TRANSMISSION SCHEDULING COORDINAITON AMONG COLLOCATED 

INTERNET RADIOS 

ABSTRACT OF THE INVENTION 

A method for collocated nodes communicating over a first interface 
5 to agree on a conflict-free transmission schedule among themselves, which 
they can then use to collaborate with neighbors accessed through a second 
interface, for example through wireless links in order to obtain collision-free 
transfers of unicast, multicast and broadcast packets over wireless channels, 
and channel access delay guarantees. The collocated nodes behave as a 
o single virtual node for the purpose of establishing a consistent transmission 
schedule throughout the nodes of a multihop wireless network. 
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